Goto

Collaborating Authors

 neural bridge sampling


Review for NeurIPS paper: Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

Neural Information Processing Systems

Summary and Contributions: Summary of contributions i) They set out to deploy probabilistic methods to determine the probability of dangerous events and determine the safety of a given, where dangerous events are simulated in a custom-built simulator, that combines exploration, exploitation, and optimization techniques to find failure modes and estimate the rate of occurrence. Summary They combine an adapted version of HMC, that they call warped HMC which, through sequential updates, utilizes normalizing flows and bridge sampling to extract samples corresponding to rare-events in a variety of different scenarios, generated via stochastic simulation. This paper shares some similar themes with NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport, but they also combine a series of other techniques. I had read this two-weeks ago and contributed to the discussions, so I apologise for the delay in the update. Just a few points and I believe the AC/ other reviewers have provided you with more feedback.


Review for NeurIPS paper: Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

Neural Information Processing Systems

The paper proposes a method for empirical verification of safety system by developing an iterative method for sampling rare and potentially out of bounds system states. The reviewers agree that the strengths are in the novel and important area that is generally little worked in. The method is well founded, rigorously analyzed, and evaluated thoroughly. The methodology presented in the paper is general and applicable to other applications other than the safety, and can generate a lot of follow-up work. In the final version the authors should: - Address R1's concerns about ReLU non-reversibility and show the validity of the bridge-sampling hybrid algorithm empirically or analytically.


Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

Neural Information Processing Systems

Learning-based methodologies increasingly find applications in safety-critical domains like autonomous driving and medical robotics. Due to the rare nature of dangerous events, real-world testing is prohibitively expensive and unscalable. In this work, we employ a probabilistic approach to safety evaluation in simulation, where we are concerned with computing the probability of dangerous events. We develop a novel rare-event simulation method that combines exploration, exploitation, and optimization techniques to find failure modes and estimate their rate of occurrence. We provide rigorous guarantees for the performance of our method in terms of both statistical and computational efficiency.